1. Field of the Invention
The present disclosure relates generally to packet switch/router design and operation, and more particularly to techniques for reducing serial channel crosstalk across an electrical backplane.
2. Description of Related Art
Packets switches/routers have external ports on which they receive and transmit packets. Most received packet traffic has a destination other than the switch/router itself. The packet/switch router determines, from a packet's headers, and possibly the port on which the packet is received, an appropriate output port or ports for each such packet. The packet is switched internally to the desired output port or ports that move the packet towards its destination.
Most switch/routers have an internal switching architecture capable of sustaining an internal packet throughput at least roughly equal to the aggregate line rate of all the external ports on the switch/router (some architectures allow an “oversubscribed” configuration that assumes all external ports are operating on average at some fraction of their line rate). 10 Gbps (Billion bits/second) ports are now becoming common, and systems are in design for even higher line rate external ports. Accordingly, the internal switching architectures for medium-to-large switch/routers handling 10 Gbps and higher-speed external ports require internal throughputs measurable in Terabits/second (Tbps) to tens of Tbps. Practically, such systems are modular, with the external ports hosted on multiple line cards that attach to one or more backplanes. Internal packet switching between line cards requires passing the packet data across a backplane.
FIG. 1 shows one prior art data plane architecture for a high-speed switch/router 100. The data plane architecture includes line cards LC0 to LCn, switch fabric cards SF0 to SFm, and a backplane 110 to which the line cards and switch fabric cards connect. Line cards can support different external signaling speeds and formats as needed for a particular application. All line card options are compatible, however, with the backplane and switch fabric cards.
Line card LC0 is representative of all line cards. Line card LC0 contains physical interface devices PHY0.0, PHY0.1, PHY0.2, packet processor devices IPP0 and EPP0, traffic manager devices ITM0 and ETM0, and serializer/deserializer (SERDES) devices (grouped together as SDL0), as well as other control plane devices (not shown) for managing the illustrated devices. Each physical interface device supports one or more external ports of a desired format and speed, to which external connections (e.g., copper cable or fiber optic cable) are made. The physical interface devices interpret the physical signaling for received packet data, and supply received packet data to an ingress packet processor device IPP0. Ingress packet processor IPP0 provides any necessary updates to each packet, determines an appropriate egress port or ports for each packet, tags the packets with an internal header, and submits them to an ingress traffic manager ITM0 for queuing. Ingress traffic manager ITM0 queues each packet with others of similar priority that are destined for the same egress line card, and notifies a central scheduler (not shown) as to which queues have data waiting. When instructed by the scheduler, packets bound for a particular egress line card are dequeued and presented to SERDES SDL0 for transmission across the backplane.
On the egress side, line card LC0 SERDES SDL0 receives packet data for packets that an ingress processor has determined have an egress port served by line card LC0. SERDES SDL0 deserializes the packet data and supplies the data to an egress traffic manager ETM0 for queuing. ETM0 supplies the packets to an egress packet processor EPP0 in a manner that attempts to treat packets of similar priority fairly. Egress packet processor EPP0 performs any necessary packet header manipulation, removes the internal tag headers, and supplies the packets to the appropriate physical interface device for transmission on an external port.
Backplane 110 connects the line cards and switch fabric cards to the system. Backplane 110 contains male connectors, e.g., CML0, aligned to connect with companion line card female connectors, e.g., connector CFL0 on line card LC0. Backplane 110 also contains male connectors, e.g., CMS0, aligned to connect with companion switch fabric card female connectors, e.g., connector CFS0 on switch fabric card SF0.
To support data plane traffic, backplane 110 contains differential trace pairs connecting designated pins of each line card male connector to each switch fabric male connector. For instance, at each line card slot, one defined connector pin pair supports transmission across a differential trace pair to switch fabric card SF0, and another defined connector pin pair supports reception across a differential trace pair from switch fabric card SF0. Other defined connector pin pairs at each line card slot support transmission and reception to other switch fabric cards, and/or additional channels to switch fabric card SF0. Likewise, at each switch fabric card slot, one defined connector pin pair supports transmission across a differential trace pair to line card LC0, and another defined connector pin pair supports reception across a differential trace pair from line card LC0. Other defined connector pin pairs at each switch fabric card slot support transmission and reception to/from other line cards, and/or additional channels to line card LC0.
Switch fabric card SF0 is representative of all switch fabric cards. Switch fabric card SF0 contains a crossbar switch CS0 and SERDES devices (grouped together as SDS0), as well as other control plane devices (not shown) for managing the illustrated devices. The SERDES devices SDS0 function like the line card SERDES devices, such that each device communicates across a backplane differential trace pair with a paired device. Each SERDES submits parallel data to the crossbar switch CS0, and receives parallel data from the crossbar switch (this is generally true, although some channels could be idle, depending on the crossbar configuration). A central scheduler (not shown) supplies the crossbar switch CS0 with a configuration schedule that describes the crossbar configurations desired to effectuate the transfer of packets from their respective ingress line cards to their respective egress line cards.
The configuration schedule changes the crossbar configuration once per epoch, where each epoch allows each scheduled line card the opportunity to transmit (as typically configured) tens of thousands of octets of packet data to a target line card. When a line card is scheduled, it selects packets from one or more queues corresponding to the scheduled destination, divides the packet data for parallel transmission to all of the switch fabric cards, and sends the packet data to the switch fabric cards at the appropriate epoch. The crossbar switches turn the packet data simultaneously to the appropriate egress channels to reach the scheduled egress line card—which receives and reassembles the packet data. Many such transfers can occur simultaneously during the same epoch, with the scheduler scheduling as many non-conflicting transfers as possible each epoch.
In one embodiment, a hybrid signaling scheme is used to send data across the backplane. Each transmitting SERDES constructs a backplane epoch frame 200, illustrated in FIG. 2. The packet headers and data supplied by the ITM to the SERDES are scrambled prior to transmission. The remainder of epoch frame 200 is transmitted using 8b/10b block coding, as follows. Each frame begins and ends with a string of 8b/10b alignment characters (see portions of epoch frame 200 labeled “SYNC”), which each SERDES transmits when it has no other data to send. When the SERDES receives a start of epoch (“SOE”) signal, it transmits some minimum number of sync characters, followed by an 8b/10b start character S. The start character is followed by scrambled packet headers/data, a cyclic redundancy check trailer CRC in 8b/10b format, and a back channel trailer BCH in 8b/10b format. After the back channel trailer BCH, the SERDES resumes transmitting alignment characters until the next SOE is received.
FIG. 3 shows a block diagram 300 for a SERDES transmit/receive pair as used in switch/router 100. Parallel data DATA IN is scrambled by a scrambler upon the command of a control block. The control block also produces the 8b/10b control and header/trailer characters, and operates a multiplexer that merges the 8b/10b characters with the scrambler output SCR for submission to a differential transmitter Tx. The control block resets the scrambler with a seed value at the beginning of each epoch. On the deserializer path, a differential receiver Rx senses data and recovers timing for a received differential signal. A SYNC block detects the 8b/10b alignment characters in the data stream and aligns the data appropriately on 8b/10b word boundaries. An inbound control block receives the 8b/10 control and header/trailer characters through a demultiplexer, and directs the scrambled data to a descrambler. The descrambler is reset from a seed register each epoch, allowing it to create descrambled DATA OUT. The inbound control block may receive back channel data BCH that comprises transmit tap coefficient instructions for Tx, which the inbound control block passes to the outbound control block. The inbound control block may also receive back channel data BCH that allows it to adjust the decision feedback equalization (DFE) taps in Rx.